Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump testcontainers to 1.16 #19648

Merged
merged 3 commits into from
Aug 26, 2021
Merged

Bump testcontainers to 1.16 #19648

merged 3 commits into from
Aug 26, 2021

Conversation

geoand
Copy link
Contributor

@geoand geoand commented Aug 25, 2021

Supersedes: #18873

See the first commit for why this works

@geoand geoand requested a review from famod August 25, 2021 09:52
@quarkus-bot quarkus-bot bot added area/dependencies Pull requests that update a dependency file area/redis labels Aug 25, 2021
@geoand geoand changed the title Bump to testcontainer 1.16 Bump testcontainers to 1.16 Aug 25, 2021
@famod
Copy link
Member

famod commented Aug 25, 2021

Good catch! Let me double-check locally in Ubuntu WSL and Windows natively...

Copy link
Member

@famod famod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A lot of duplication in those processors but I guess that could be some refactoring task for later.

What I find very interesting is that with latest Docker Desktop 3.6.0, I cannot run those invoker tests on main anymore (both in WSL2 and Windows natively) because of:

Can not connect to Ryuk at localhost:49156: java.net.ConnectException: Connection refused: connect

I'm pretty sure I wasn't seeing that with 3.5.x.

But with this change, I can run those tests again! So I propose to backport this, ideally even to 2.1 if feasible.

@famod famod added the triage/waiting-for-ci Ready to merge when CI successfully finishes label Aug 25, 2021
@famod
Copy link
Member

famod commented Aug 25, 2021

PS: I was surprised start-containers profile is disabled on Windows. This might have been needed in CI at some point (GH windows runner does not support Linux containers), but AFAICS Windows jobs are not even using -Dstart-containers. (/cc @gsmet)

When I force-enable that profile on Windows, then the surefire part runs fine, but failsafe hangs very early on.
jstack showed this:

"main" #1 prio=5 os_prio=0 cpu=6578.13ms elapsed=305.94s tid=0x000001dbf35f5000 nid=0x23e4 runnable  [0x0000009cebefd000]
   java.lang.Thread.State: RUNNABLE
        at java.io.FileInputStream.readBytes([email protected]/Native Method)
        at java.io.FileInputStream.read([email protected]/FileInputStream.java:279)
        at java.io.BufferedInputStream.read1([email protected]/BufferedInputStream.java:290)
        at java.io.BufferedInputStream.read([email protected]/BufferedInputStream.java:351)
        - locked <0x00000006d7c588e8> (a java.io.BufferedInputStream)
        at java.io.FilterInputStream.read([email protected]/FilterInputStream.java:107)
        at java.util.Properties$LineReader.readLine([email protected]/Properties.java:502)
        at java.util.Properties.load0([email protected]/Properties.java:418)
        at java.util.Properties.load([email protected]/Properties.java:407)
        - locked <0x00000006d7c5a9f0> (a java.util.Properties)
        at org.codehaus.plexus.languages.java.jpms.MainClassModuleNameExtractor.extract(MainClassModuleNameExtractor.java:102)
        at org.codehaus.plexus.languages.java.jpms.LocationManager.resolvePaths(LocationManager.java:234)
        at org.apache.maven.plugin.surefire.AbstractSurefireMojo.newStartupConfigWithModularPath(AbstractSurefireMojo.java:2011)
        at org.apache.maven.plugin.surefire.AbstractSurefireMojo.createStartupConfiguration(AbstractSurefireMojo.java:1889)
        at org.apache.maven.plugin.surefire.AbstractSurefireMojo.createForkStarter(AbstractSurefireMojo.java:2374)
        at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1310)
        at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1159)
        at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:932)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:957)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:289)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:193)
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0([email protected]/Native Method)
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke([email protected]/NativeMethodAccessorImpl.java:62)
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke([email protected]/DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke([email protected]/Method.java:566)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)

@geoand
Copy link
Contributor Author

geoand commented Aug 25, 2021

Thanks for checking!

Yup, there is plenty of duplication, but that's the price we pay for dev services.
It is probably possible to refactor some of the common stuff and we should likely do it at some point.

As for backporting, I dunno really... The second commit should likely be backported, but the version bump I don't know ...

I'll leave it up to @gsmet to decide

@quarkus-bot
Copy link

quarkus-bot bot commented Aug 25, 2021

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building 236900a

Status Name Step Test failures Logs Raw logs
JVM Tests - JDK 11 Build Test failures Logs Raw logs
JVM Tests - JDK 16 Build Test failures Logs Raw logs
MicroProfile TCKs Tests Verify Test failures Logs Raw logs

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 integration-tests/container-image/maven-invoker-way/target/it/container-image-jib-with-redis

org.acme.redis.IncrementResourceIT.testRedisOperations - More details - Source on GitHub

java.lang.RuntimeException: 
java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.redis.client.deployment.DevServicesRedisProcessor#startRedisContainers threw an exception: org.testcontainers.containers.ContainerLaunchException: Container startup failed
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:334)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:315)
	at io.quarkus.redis.client.deployment.DevServicesRedisProcessor.lambda$startContainer$1(DevServicesRedisProcessor.java:160)
	at java.base/java.util.Optional.orElseGet(Optional.java:369)
	at io.quarkus.redis.client.deployment.DevServicesRedisProcessor.startContainer(DevServicesRedisProcessor.java:168)
	at io.quarkus.redis.client.deployment.DevServicesRedisProcessor.startRedisContainers(DevServicesRedisProcessor.java:90)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invok...

⚙️ JVM Tests - JDK 16 #

📦 integration-tests/container-image/maven-invoker-way/target/it/container-image-jib-with-redis

org.acme.redis.IncrementResourceIT.testRedisOperations - More details - Source on GitHub

java.lang.RuntimeException: 
java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.redis.client.deployment.DevServicesRedisProcessor#startRedisContainers threw an exception: org.testcontainers.containers.ContainerLaunchException: Container startup failed
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:334)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:315)
	at io.quarkus.redis.client.deployment.DevServicesRedisProcessor.lambda$startContainer$1(DevServicesRedisProcessor.java:160)
	at java.base/java.util.Optional.orElseGet(Optional.java:364)
	at io.quarkus.redis.client.deployment.DevServicesRedisProcessor.startContainer(DevServicesRedisProcessor.java:168)
	at io.quarkus.redis.client.deployment.DevServicesRedisProcessor.startRedisContainers(DevServicesRedisProcessor.java:90)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invok...

⚙️ MicroProfile TCKs Tests #

📦 tcks/microprofile-fault-tolerance

org.eclipse.microprofile.fault.tolerance.tck.TimeoutUninterruptableTest.testTimeoutAsyncBulkhead line 190 - More details - Source on GitHub

java.lang.AssertionError: Unexpected exception thrown from Future
	at org.testng.Assert.fail(Assert.java:85)
	at org.eclipse.microprofile.fault.tolerance.tck.util.Exceptions.expect(Exceptions.java:98)
	at org.eclipse.microprofile.fault.tolerance.tck.TimeoutUninterruptableTest.testTimeoutAsyncBulkhead(TimeoutUninterruptableTest.java:190)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at io.quarkus.arquillian.QuarkusProtocol$QuarkusMethodExecutor$1.invoke(QuarkusProtocol.java:87)
	at org.jboss.arquillian.container.test.impl.execution.LocalTestExecuter.execute(LocalTestExecuter.java:57)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base...

@geoand
Copy link
Contributor Author

geoand commented Aug 25, 2021 via email

geoand added 3 commits August 25, 2021 21:28
…tainers

This is only applicable for containers running on a shared network.
Essentially we now ensure that for containers whose status can only
be determined by making a network call, that testcontainers can make
that network call.
In testcontainers 1.15.x, the library would actually not fail
if the connection could not be made, but would essentially give
up after a while.
With this change, startup of the tests is actually faster since testcontainers
actually determines the running status much faster.
In 1.16 the behavior seems to have changed and the testcontainers seems to try
to launch new containers and never gives up.
@quarkus-bot
Copy link

quarkus-bot bot commented Aug 25, 2021

Failing Jobs - Building 0e69d68

Status Name Step Test failures Logs Raw logs
JVM Tests - JDK 11 Build Test failures Logs Raw logs
✔️ JVM Tests - JDK 16

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 extensions/smallrye-reactive-messaging-kafka/deployment

io.quarkus.smallrye.reactivemessaging.kafka.deployment.testing.KafkaDevServicesContinuousTestingTestCase.testContinuousTestingScenario1 line 64 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: expected: <1> but was: <0>
	at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55)
	at org.junit.jupiter.api.AssertionUtils.failNotEqual(AssertionUtils.java:62)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:166)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:161)
	at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:611)
	at io.quarkus.smallrye.reactivemessaging.kafka.deployment.testing.KafkaDevServicesContinuousTestingTestCase.testContinuousTestingScenario1(KafkaDevServicesContinuousTestingTestCase.java:64)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.pla...

io.quarkus.smallrye.reactivemessaging.kafka.deployment.testing.KafkaDevServicesContinuousTestingWorkingAppPropsTestCase.testContinuousTestingScenario3 - More details - Source on GitHub

java.lang.RuntimeException: 
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.kafka.client.deployment.DevServicesKafkaProcessor#startKafkaDevService threw an exception: org.testcontainers.containers.ContainerLaunchException: Container startup failed
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:334)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:315)
	at io.quarkus.kafka.client.deployment.DevServicesKafkaProcessor.lambda$startKafka$5(DevServicesKafkaProcessor.java:222)
	at java.base/java.util.Optional.orElseGet(Optional.java:369)
	at io.quarkus.kafka.client.deployment.DevServicesKafkaProcessor.startKafka(DevServicesKafkaProcessor.java:230)
	at io.quarkus.kafka.client.deployment.DevServicesKafkaProcessor.startKafkaDevServ...

@geoand geoand merged commit 6299721 into quarkusio:main Aug 26, 2021
@quarkus-bot quarkus-bot bot removed the triage/waiting-for-ci Ready to merge when CI successfully finishes label Aug 26, 2021
@quarkus-bot quarkus-bot bot added this to the 2.3 - main milestone Aug 26, 2021
@geoand geoand deleted the tc-116 branch August 26, 2021 04:10
@famod
Copy link
Member

famod commented Sep 17, 2021

What I find very interesting is that with latest Docker Desktop 3.6.0, I cannot run those invoker tests on main anymore (both in WSL2 and Windows natively) because of:

Can not connect to Ryuk at localhost:49156: java.net.ConnectException: Connection refused: connect

I'm pretty sure I wasn't seeing that with 3.5.x.

But with this change, I can run those tests again! So I propose to backport this, ideally even to 2.1 if feasible.

FTR, this is a general Docker Desktop issue on Windows: docker/for-win#11584

testcontainers 1.16 does try to work around that problem via testcontainers/testcontainers-java#4263 but it's not like nothing works anymore with <1.16, so a backport would be nice but not at all critical.

<testcontainers.version>1.15.3</testcontainers.version>
<docker-java.version>3.2.8</docker-java.version> <!-- must be the version Testcontainers use -->
<testcontainers.version>1.16.0</testcontainers.version>
<docker-java.version>3.2.11</docker-java.version> <!-- must be the version Testcontainers use -->
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want it to be the version testcontainers uses, consider not specifying it directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/dependencies Pull requests that update a dependency file area/devservices area/redis
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants